This invention relates to a display apparatus such as, for example, a head mounted display (HMD) apparatus or a television receiver which reproduces and displays a video signal in the form of a digital signal transmitted and received as a compressed signal used, for example, in satellite broadcasting and/or outputs sound, and more particularly to a display apparatus which calculates a turning angle, an amount of movement of the head of a viewer in the direction of the visual point and so forth, extracts and reproduces an image supplied with the aspect ratio of 16:9 or 4:3 on the real time taking an amount of movement of the image and processes audio information in response to the amount of movement to provide the feeling of presence to the viewer at a comparatively low cost.
In the field of the image technology of AV (audio-visual) apparatus or AV systems, an image reproduction method is conventionally known wherein an image being observed by a viewer is varied in response to a turning movement of the head or some other movement of the viewer.
Meanwhile, the MPEG2 format of a data compression system for processing an image of a moving object have been standardized in the world, and data encoders and data decoders conforming to the standards have been marketed already. An MPEG decoder has a PAN & SCAN function. This function writes region designating information representative of an aspect ratio of a display screen in a header part of a video signal defined on the MPEG2 decoder to designate an aspect ratio of a display image.
Making use of this function, it is possible to extract image information of the aspect ratio of 4:3 from compressed image information of the aspect ratio of 16:9 of the MPEG2 format.
On the other hand, in the field of the audio technology of AV apparatus, a method is known wherein, as disclosed in Japanese Patent Laid-Open Application No. Heisei 7-95698 entitled "Audio Reproduction Apparatus" assigned to the same assignee of the present invention, when sound collected in stereo is to be reproduced by means of a headphone, the left and right acoustic outputs of the headphone are corrected with impulse responses stored in a memory in advance in response a turning angle of the head of a listener.
Further, as a sensor for detecting a movement amount or a turning angle of the body of a person, various sensors are known including such a digital angle detector which makes use of a horizontal component of force of the terrestrial magnetism as disclosed in Japanese Patent Laid-Open Application No. Heisei 7-95698 mentioned hereinabove, and a sensor which employs a gyro.
However, according to the conventional image processing apparatus described above, in principle, a turning angle of the head of a person is detected by means of a sensor, and a special video signal which corresponds to a number of images several to several tents times a display screen of an AV apparatus is processed in synchronism with the speed of the turning movement based on the turning angle information from the sensor. Consequently, a very large memory capacity and a high processing speed are demanded, and this makes an obstacle to minimization and reduction in weight and in cost of an AV apparatus, which is a first problem of the conventional image processing method.
It is a second problem of the conventional image processing apparatus that a displacement is produced between normal positions of a video image and an audio image. In particular, where a fixed type image reproduction apparatus wherein an image is not varied by a movement of the head of a person like a television receiver or the like having an ordinary CRT display tube is used in combination with a headphone wherein the normal position of an audio image is varied by a movement of the head of a person, since only the normal position of the audio image is varied in response to a movement of the head, depending upon the accuracy or the like of a gyro sensor which detects the turning angle of the head, a displacement is possibly produced between the position of a video image and the normal position of the audio image.
Also where a head mounted display unit wherein a video image is not varied by a movement of the head is used in combination with a headphone of the type described above, when the head is moved, the video image is not varied, and consequently, the position of the video image is displaced from the normal position of the audio image.
It is to be noted that the normal position of an audio image signifies the position of a virtual sound source formed in the head of a viewer when the viewer listens to an audio signal originally adjusted so as to be reproduced from an audio outputting apparatus such as a loudspeaker or the like using a headphone, such as, for example, a front right position, a front left position or a front central position.